An Explanation of the Expectation Maximization Algorithm, Report no. LiTH-ISY-R-2915
نویسنده
چکیده
The expectation maximization (EM) algorithm computes maximum likelihood estimates of unknown parameters in probabilistic models involving latent variables. More pragmatically speaking, the EM algorithm is an iterative method that alternates between computing a conditional expectation and solving a maximization problem, hence the name expectation maximization. We will in this work derive the EM algorithm and show that it provides a maximum likelihood estimate. The aim of the work is to show how the EM algorithm can be used in the context of dynamic systems and we will provide a worked example showing how the EM algorithm can be used to solve a simple system identi cation problem.
منابع مشابه
Maximum Likelihood Estimation in Mixed Linear/Nonlinear State-Space Models, Report no. LiTH-ISY-R-2958
The primary contribution of this paper is an algorithm capable of identifying parameters in certain mixed linear/nonlinear state-space models, containing conditionally linear Gaussian substructures. More speci cally, we employ the standard maximum likelihood framework and derive an expectation maximization type algorithm. This involves a nonlinear smoothing problem for the state variables, whic...
متن کاملParticle Filter Approach to Nonlinear System Identification under Missing Observations with a Real Application, Report no. LiTH-ISY-R-2895
This article reviews authors' recently developed algorithm for identi cation of nonlinear state-space models under missing observations and extends it to the case of unknown model structure. In order to estimate the parameters in a state-space model, one needs to know the model structure and have an estimate of states. If the model structure is unknown, an approximation of it is obtained using ...
متن کاملMaximum Likelihood Estimation of Gaussian Models with Missing Data—Eight Equivalent Formulations, Report no. LiTH-ISY-R-3013
In this paper we derive the maximum likelihood problem for missing data from a Gaussian model. We present in total eight di erent equivalent formulations of the resulting optimization problem, four out of which are nonlinear least squares formulations. Among these formulations are also formulations based on the expectation-maximization algorithm. Expressions for the derivatives needed in order ...
متن کاملSystem Identification of Nonlinear State-Space Models, Report no. LiTH-ISY-R-2977
This paper is concerned with the parameter estimation of a general class of nonlinear dynamic systems in state-space form. More speci cally, a Maximum Likelihood (ML) framework is employed and an Expectation Maximisation (EM) algorithm is derived to compute these ML estimates. The Expectation (E) step involves solving a nonlinear state estimation problem, where the smoothed estimates of the sta...
متن کاملEstimating State-Space Models in Innovations Form using the Expectation Maximisation Algorithm, Report no. LiTH-ISY-R-3002
The expectation maximisation (EM) algorithm has proven to be e ective for a range of identi cation problems. Unfortunately, the way in which the EM algorithm has previously been applied has proven unsuitable for the commonly employed innovations form model structure. This paper addresses this problem, and presents a previously unexamined method of EM algorithm employment. The results are pro le...
متن کامل